Skip to content

Conversation

Copy link

Copilot AI commented Oct 19, 2025

This PR implements docker-style progress bars and resumable download functionality for the llama-server -dr flag, which downloads models from Docker Hub using OCI protocol.

Problem

Previously, when using llama-server -dr to download models from Docker Hub:

  • No progress indication - users had no visibility into download status
  • No resume capability - interrupted downloads had to restart from scratch, wasting bandwidth and time
  • No integrity verification on cached files

Solution

1. Docker-Style Progress Bars

Added real-time progress display showing:

  • Layer digest (short form: first 12 chars)
  • Visual progress bar (50 characters wide)
  • Percentage complete
  • Downloaded size / Total size in MB
  • Current download speed in MB/s

Progress updates every 100ms for smooth visual feedback and automatically detects TTY to only show on interactive terminals.

Example:

1234567890ab: Downloading [===================>              ] 39.0% (39.00 MB / 100.00 MB) 20.10 MB/s

2. Resumable Downloads

Implemented automatic resume capability:

  • Partial downloads saved with .tmp extension
  • Layer digest stored in .digest file for integrity verification
  • Automatically resumes interrupted downloads from last position
  • Efficiently skips already-downloaded bytes using io.CopyN
  • Verifies file size matches expected size before finalizing
  • Handles digest changes (model updates) by starting fresh download

Example resume:

1234567890ab: Resuming download from 39.00 MB
1234567890ab: Downloading [===================>              ] 41.0% (41.00 MB / 100.00 MB) 18.50 MB/s

3. Enhanced Cache Validation

  • Validates cached models using stored digest
  • Detects and handles digest mismatches
  • Cleans up invalid partial downloads
  • Uses atomic file operations (rename from .tmp to final)

Technical Details

Implementation

  • Added progressWriter struct in Go that wraps io.Writer
  • Uses sync/atomic for thread-safe progress tracking
  • Progress written to stderr to not interfere with program output
  • Progress updates throttled to 100ms intervals to avoid excessive I/O

Files Modified

  • oci-go/oci.go (+180 lines): Core implementation
  • docs/oci-registry.md (+43 lines): Documentation updates

Code Quality

  • ✅ Go fmt applied
  • ✅ Go vet passed
  • ✅ Code review passed - no issues
  • ✅ CodeQL security scan passed - no vulnerabilities
  • ✅ Builds successfully on Linux x86_64

Benefits

  1. Better User Experience: Users can see download progress and estimate completion time
  2. Saves Bandwidth: Resume capability prevents re-downloading large files
  3. Saves Time: Interrupted downloads can be resumed instantly
  4. Better Reliability: Digest verification ensures file integrity
  5. Familiar Interface: Progress format similar to Docker pulls

Backward Compatibility

Fully backward compatible:

  • No breaking changes to existing functionality
  • Works with existing cache files
  • Handles both new and old partial downloads gracefully
  • No changes to command-line interface

Usage

No changes required - existing -dr flag now includes progress and resume:

# Download with progress bar (interruptible and resumable)
./llama-server -dr ai/smollm2:135M-Q4_0

# Works with all existing -dr options
./llama-cli --docker-repo myuser/model:tag

Warning

Firewall rules blocked me from connecting to one or more addresses (expand for details)

I tried to connect to the following addresses, but was blocked by firewall rules:

  • ggml.ai
    • Triggering command: /home/REDACTED/work/llama.cpp/llama.cpp/build/bin/test-arg-parser (dns block)

If you need me to access, download, or install something from one of these locations, you can either:

Original prompt

Add docker-style progress bars when pulling via "llama-server -dr". Also add resumable downloads functionality to "llama-server -dr".


✨ Let Copilot coding agent set things up for you — coding agent works faster and does higher quality work when set up for your repo.

ericcurtin and others added 2 commits October 19, 2025 15:42
So we can pull from any OCI registry, add authentication, etc.

Signed-off-by: Eric Curtin <[email protected]>
Copilot AI changed the title [WIP] Add docker-style progress bars and resumable downloads Add docker-style progress bars and resumable downloads to llama-server -dr Oct 19, 2025
Copilot AI requested a review from ericcurtin October 19, 2025 15:05
Copilot finished work on behalf of ericcurtin October 19, 2025 15:05
@ericcurtin ericcurtin force-pushed the integrate-go-containerregistry-library branch 13 times, most recently from e4b192d to 2935963 Compare October 19, 2025 20:47
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants